Re-examining the Effectiveness of Manual Review
نویسنده
چکیده
Recent studies have found that automated retrieval methods in ediscovery are not only cheaper than manual review, but are also as or more reliable. We argue that these studies, while suggestive, are not conclusive. There is a high variability in the quality of unsupervised manual reviewers, as data from the TREC Legal Track shows. The best manual reviewers are as reliable as automated methods, and a properly supervised manual review may be more reliable than automation. We show the effectiveness of the simple review management approach of observing the proportions found relevant between reviewers. Finally, we describe the experimental protocol necessary for a more conclusive comparison of manual and automated review.
منابع مشابه
Health Technology Assessment of the Psychotherapy Effect on Prevention of Suicide and Self-Harm: A Rapid Review of Secondary Studies
Introduction: According to the World Health Organization (WHO) report, acute depression disorders are the second largest health problem in the world which can cause disability and imposes huge costs on individuals and societies. This study aims to investigate the effectiveness and cost-effectiveness of using psychotherapy in patients with depression for prevention of suicide an...
متن کاملComparing the effectiveness of strengthening working memory and critical thinking training on emotion regulation strategies of children with special learning disorder
Abstract Objective: To compare the effectiveness of memory work and critical thinking training of children's movement regulation with special methods. Method: The current research was a semi-experimental research with a pre-test, post-test and control group design. The statistical population included all students aged 9 to 12 who referred to the center for special problems in Tabriz distri...
متن کاملA systematic review of the effectiveness of exercise, manual therapy, electrotherapy, relaxation training, and biofeedback in the management of temporomandibular disorder.
BACKGROUND AND PURPOSE This systematic review analyzed studies examining the effectiveness of various physical therapy interventions for temporomandibular disorder. METHODS Studies met 4 criteria: (1) subjects were from 1 of 3 groups identified in the first axis of the Research Diagnostic Criteria for Temporomandibular Disorders, (2) the intervention was within the realm of physical therapist...
متن کاملWhat constitutes effective manual handling training? A systematic review.
BACKGROUND Injuries caused by manual handling are a major burden to society. Manual handling training programmes have been designed to reduce the likelihood of injury among the workforce; however, concerns have been raised over the efficacy of current manual handling training methods. AIMS To undertake a systematic review of the literature examining the effectiveness of different approaches t...
متن کاملCost-effectiveness Analysis of Hospital Information System in Comparison with Manual System in Selected Sections of Namazi Hospital of Shiraz
Background & objectives: Health information technology provides a powerful tool for monitoring the health system's performance. This study aimed to assess the cost-effectiveness of hospital information system in comparison with manual system in selected sections of Namazi hospital in Shiraz. Methods: This descriptive-analytical research was a economic evaluation carried out at Namazi hospital ...
متن کامل